In processing a digital image, it is common to sharpen the image to enhance fine detail with sharpening algorithms. Typically, sharpening is performed by a convolution process (for example, see A. K. Jain, Fundamentals of Digital Image Processing, Prentice-Hall: 1989, pp. 249-251). The process of unsharp masking is an example of a convolution-based sharpening process. For example, sharpening an image with unsharp masking can be described by the equation:s(x,y)=i(x,y)**b(x,y)+βf(i(x,y)−i(x,y)**b(x,y))   (0)where:                s(x,y)=output image with enhanced sharpness        i(x,y)=original input image        b(x,y)=lowpass filter        β=unsharp mask gain factor        f( )=fringe function        ** denotes two dimensional convolution        (x,y) denotes the xth row and the yth column of an image        
Typically, an unsharp image is generated by convolution of the image i(x,y) with a lowpass filter b(x,y) (i.e., the unsharp image is given by i(x,y)**b(x,y)). Next, a highpass, or fringe image is generated by subtracting the unsharp image from the original image (i.e., the highpass data is found with i(x,y)−i(x,y)**b(x,y)). This highpass data is then modified by either a gain factor β or a fringe function f( ) or both. Finally, the modified highpass data is summed with either the original image or the unsharp image to produce a sharpened image.
A similar sharpening effect can be achieved by modification of the image in a frequency domain (e.g., a Fourier domain) as is well known in the art of digital signal processing.
Digital images come in a variety of sizes (the number of rows and columns of pixels in an image describes the size of the image; also commonly referred to as resolution.) The size of digital image depends on many factors, including the inherent resolution of the image source (e.g. digital camera image sensor), digital camera settings, and whether and to what extent cropping or interpolation has been applied to the digital image.
When prints are generated from a digital image, generally the size of the print is specified. Common photographic print sizes are 4×6 inches, 8×10 inches, and 5×7 inches. Therefore, the magnification of the pixels of the digital image is dependent on the desired print size. The spatial area covered on the print by each pixel of the digital image is dependent on the print size.
If a digital sharpening process is applied without knowing the magnification required to generate a print, there is no way of knowing which spatial frequencies of the print will be affected. Therefore, sharpening the digital image without knowing the size of the desired print results in sub-optimal quality.
One solution to this problem is to resize the digital image using interpolation to a normalized size, such that spatial area covered on the print by each pixel of the resized digital image is standard. Then a standard sharpening is applied to resized digital image in preparation for printing. Generally, this process results in increasing the resolution of most digital images through interpolation. This in turn requires that more pixels be sharpened, which is inefficient.
Another problem associated with sharpening digital images is that sharpening amplifies noise and other artifacts—e.g. the artifacts associated with JPEG compression, which is commonly used to reduce the file size of an image.
It is occasionally desirable to sharpen different regions or pixels of the image by different amounts. For example, is it has been suggested that it is desirable to sharpen the pixels representing human faces to a lesser degree than pixels representing a building. For example, in U.S. Pat. No. 5,682,443 Gouch et al. describe the modification of the gain of the unsharp mask based on the color of a pixel (and the color of the surrounding neighborhood). Gouch did not describe sharpening of an image taking into account the magnification. In addition, Gouch's sharpening method is not designed to minimize the amplification of compression artifacts.
Alternatively, in U.S. Pat. No. 4,571,635, Mahmoodi and Nelson teach a method of deriving a gain factor β that is used to scale the high frequency information of the digital image depending on the standard deviation of the image pixels within a neighborhood. In addition, in U.S. Pat. No. 5,081,692, Kwon and Liange teach that a gain factor β is based on a center weighted variance calculation. In U.S. Pat. No. 4,761,819, Denison describes a method where the gain factor of an unsharp mask is dependent on both a local variance calculation and a noise statistic. While these methods do indeed sharpen the image while attempting to minimize noise amplification, they are computationally complex. In addition, neither describes sharpening an image taking into account the magnification, or sharpening while minimizing the amplification of compression artifacts.
Shimazaki in U.S. Pat. No. 5,051,842 describes an apparatus which generates unsharp signals from images, derives two parameters based on either the image signal level or the unsharp signal level from a pre-determined lookup table, multiplies one parameter with the image signal, multiplies the other parameter with the unsharp signal, and adds the two resulting signals to obtain the final image signal. One embodiment requires that the sum of the two parameters equal one for all image signal levels. In this case, the method is mathematically equivalent to the unsharp mask equation. Shimazaki teaches that the two parameters are signal dependent with the signals representing image highlights resulting in the highest degree of sharpening. The two parameters are chosen such that the sharpening decreases as either the image signal or the unsharp signal decreases until the sharpening level is zero. At that point, the sharpening converts to blurring as the image signal or unsharp signal continue to decrease into the shadow region of the density range. Shimazaki did not describe sharpening of an image taking into account the magnification. In addition, Shimazaki's sharpening method is not designed to minimize the amplification of compression artifacts.
Gallagher and Gindele, in U.S. Pat. No. 6,167,165 describe a method of selecting a gain for an unsharp mask based on a local intensity level. Gallagher does not describe sharpening of an image taking into account the magnification. In addition, Gallagher's sharpening method is not designed to minimize the amplification of compression artifacts.
Keyes and Hoff, in U.S. Pat. No. 6,091,861 describe a method of selecting a constant, position independent gain factor based on the exposure of the image. Lower gain factors will be selected for images that are underexposed, thereby providing less gain for that image than for a normally exposed image. Keyes describes sharpening of an image by setting a sharpening amount based in part on the image magnification. However, the Keyes method only varies the gain of a fixed filter. Unfortunately, this method produces sub-optimal effects for different spatial frequencies because the frequency response of the lowpass filter is constant. Prints of equal size made from digital images having different numbers of rows and columns of pixels would have an inconsistent look. This results from the frequency response of the blurring filter being fixed with respect to cycles per pixel, rather than being fixed with respect to a physical property of the print itself (e.g cycles per mm on the print, or cycles per degree.) In addition, the Keyes sharpening method is not designed to minimize the amplification of compression artifacts.
Inoue and Tajima describe an adaptive method of determining a gain factor by analyzing the image and searching for edges in the article “Adaptive Image Sharpening Method Using Edge Sharpness,” IEICE Transactions on Information and Systems, vol. E76-D, no. 10, p. 1174-80, October 1993. The authors describe a sharpening where the spatial frequency characteristics of a filter are properly described in terms of spatial frequencies (cycles/degree) (see FIG. 4 and Equation 6 of the article). However, the authors do not describe an efficient method for generating the filter for images having various numbers of rows and column of pixels. In addition, the method is not designed to minimize the amplification of compression artifacts.
In U.S. Pat. No. 6,222,173, Meeussen describes a method of sharpening and resizing images to prevent the occurrence of images that are not crispy. The method involves combining a sharpening filter with the interpolation filter before application to the image. The method does not ensure that the image sharpening properly enhances the desired spatial frequencies of an output print. In addition, the method does not prevent the amplification of compression artifacts.